Method and/or apparatus for frame accurate hot failover

ABSTRACT

A method for switching between two redundant bitstreams. The first bitstream may be presented to a first pipeline. The second bitstream may be presented to a second pipeline. The first bitstream and the second bitstream may contain redundant information received from independent sources. If the first bitstream fails, the method may present an output of the second pipeline to the output pipeline. Data in a buffer in the second pipeline may be used to pass a next frame to the output pipeline. A size of a buffer of the first pipeline and a size of the buffer in the second pipeline may be adjusted based on a time of reception of the first and the second bitstream.

CROSS-REFERENCES TO RELATED APPLICATIONS

This application is a continuation of allowed U.S. application Ser. No.15/332,952, filed Oct. 24, 2016, which is a continuation of U.S.application Ser. No. 14/487,318, filed Sep. 16, 2014, now U.S. Pat. No.9,479,720, issued Oct. 25, 2016, both entitled “METHOD AND/OR APPARATUSFOR FRAME ACCURATE HOT FAILOVER,” of which the full disclosures of theseapplications are incorporated herein by reference for all purposes.

FIELD OF THE INVENTION

The present invention relates to video processing generally and, moreparticularly, to a method and/or architecture for frame accurate hotfailover.

BACKGROUND OF THE INVENTION

Conventional live transcoding systems accept multimedia inputs (i.e.,inputs containing video, audio, captions, etc.) in real time. Suchsystems transcode those inputs in one of many ways (for instance, bychanging the video compression) and generate streams presented either toa file or to a live output. The operator of such a system often hascustomers and/or advertisers that rely on the output of the livetranscoding system. The stability of the output is imperative. To helpensure a reliable output, some live transcoding systems accept multipleredundant inputs (for instance, the same multimedia content delivered asa MPEG2 transport stream over user datagram protocol (UDP) on twodifferent network interface controllers (NICs)). Failover from one input(the active input) to the other (the standby input) occurs if a problemis detected in the former. The failover from one input to the next isusually detectable on the output in one of many undesirable ways. In thebest case, an input freeze or black picture is created. In the worstcase, the output transport stream is totally corrupted.

In conventional systems, the failover transition is not frame accurate.After the failover, the video and audio content does not resume with thenext frame that would have been output had a failover not occurred. Lackof frame accuracy results in either a segment of the input content neverreaching the output or in retransmission of video and audio frames thathave already been transmitted on the output. The problem is exacerbatedwhen the active and standby inputs arrive across different networks (toavoid a network failure from interrupting both inputs). Differentnetworks often have different delays. One input may be delayedconsiderably relative to the other.

Another problem with conventional systems is that the failovertransition from one input to the next input takes time which results inan underflow of the output buffer. Such delay produces a time gap in thedelivery of output packets. When generating an output for a file, such adelay is often not a concern. When generating a real-time output, likeMPEG2 transport streams over UDP, gaps on the output can cause playoutissues with down stream decoders, which is undesirable.

It would be desirable to implement frame accurate hot failover in areal-time video transcoding system.

SUMMARY OF THE INVENTION

The present invention concerns a method for switching between tworedundant bitstreams comprising the steps of (A) receiving a firstbitstream and a second bitstream, where (i) the first bitstream may bepresented to a first pipeline and the second bitstream may be presentedto a second pipeline and (ii) the first bitstream and the secondbitstream may contain redundant information received from independentsources and (iii) an output of the first pipeline may be presented to anoutput pipeline, (B) processing the first bitstream in the firstpipeline and the second bitstream in the second pipeline and (C) if thefirst bitstream fails, presenting an output of the second pipeline tothe output pipeline. Data in a buffer in the second pipeline may be usedto pass a next frame to the output pipeline. A size of a buffer of thefirst pipeline and a size of the buffer of the second pipeline may beadjusted based on a time of reception of the first and the secondbitstream.

The objects, features and advantages of the present invention includeproviding a hot failover method and/or device that may (i) be frameaccurate, (ii) avoid overflow and underflow when switching, (iii)produce a fast failover, (iv) align frames of each source usingtimecodes, (v) avoid visual gaps in the output stream and/or (vi) becost effective to implement.

BRIEF DESCRIPTION OF THE DRAWINGS

These and other objects, features and advantages of the presentinvention will be apparent from the following detailed description andthe appended claims and drawings in which:

FIG. 1 is a diagram of a context of the present invention;

FIG. 2 is a block diagram of the present invention;

FIG. 3 is a more detailed diagram of the present invention;

FIG. 4 is a flow diagram of an overall process of the invention;

FIG. 5 is a flow diagram of a process showing the detection of the timecode and delivery of the next frame during a failure;

FIG. 6 is a flow diagram of a process showing the re-sizing of the delaybuffers;

FIG. 7a is a diagram showing buffer occupancy when the arrival of framesis matched;

FIG. 7b is a diagram showing buffer occupancy when the arrival of framesin an active input pipeline leads the arrival of frames in a hot standbypipeline;

FIG. 7c is a diagram showing buffer occupancy when the arrival of framesin an active input pipeline lags the arrival of frames in a hot standbypipeline; and

FIG. 8 is a diagram showing an alternate embodiment of the presentinvention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

The invention provides frame accurate hot failover in a live transcodingsystem for transitioning from one multimedia input to an alternatemultimedia input. The transition may occur without observableaudio-visual discontinuities and/or timing anomalies on the output. Theinvention may provide failover from one input to the other that is bothframe accurate and/or that produces no gaps in the output during thefailover transition.

Referring to FIG. 1, a block diagram of a system 50 is shown providing acontext of the present invention. The system 50 is shown comprising afirst source (e.g., SOURCE 1), a second source (e.g., SOURCE 2), anapparatus (or device) 100, and a network 60. The signal SOURCE_1 and thesignal SOURCE_2 may be redundant signals. The redundant signals SOURCE_1and SOURCE_2 may be received from independent sources. The independentsources may be from different origins (e.g., satellite, fiber, etc.). Insome embodiments, the independent sources may be from the same origin(e.g., the same origin sends redundant data through alternate networkpaths). The apparatus 100 may generate a number of streams 110 a-110 npresented to the network 60. The network 60 may be a content deliverynetwork (CDN). The network 60 may deliver multimedia content (e.g.,video) to a number of devices 70 a-70 n. In one example, the apparatus100 may be implemented as a hardware appliance (or device). In anotherexample, the apparatus 100 may be implemented as a software processand/or operation implemented on a shared processing environment (e.g.,Amazon Cloud, etc.).

Referring to FIG. 2, a block diagram of the apparatus 100 is shown inaccordance with an embodiment of the present invention. The apparatus100 generally comprises a block (or circuit) 120 and a block (orcircuit) 130. The circuit 120 may be implemented as a memory. Thecircuit 130 may be implemented as a processor. The processor 130 may beconfigured to execute computer readable instructions that, whenexecuted, cause the processor 130 to perform various steps. The memory120 includes a block 132 that may be implemented as a failover controllogic. The apparatus 100 may have an input 140, an input 142 and anoutput 144. The input 140 may receive the signal SOURCE 1. The input 142may receive the signal SOURCE 2. The output 144 may present a signal(e.g., OUTPUT). The signal OUTPUT may be generated based on either thesignal SOURCE_1 or SOURCE_2 at any moment in time, but may becontributed to by each over time (e.g., once a failover occurs).

Referring to FIG. 3, a more detailed diagram of the apparatus 100 isshown. The memory 120 is shown comprising a pipeline 160, a pipeline 162and a pipeline 170. The pipeline 160 may be implemented as an activeinput pipeline. The pipeline 162 may be implemented as a hot standbyinput pipeline. The pipeline 170 may be implemented as an outputpipeline. The pipeline 160 generally comprises a block (or circuit) 180,a block (or circuit) 182 and a block (or circuit) 184. The circuit 180may be implemented as a demultiplexing circuit. The circuit 182 may beimplemented as a decoder circuit. The circuit 184 may be implemented asa delay buffer. Similarly, the pipeline 162 may have a block (orcircuit) 180′, a block (or circuit) 182′ and a block (or circuit) 184′.The circuits 180′, 182′ and 184′ may have similar implementations as thecircuits 180, 182 and 184. The output pipeline 170 generally comprises ablock (or circuit) 190 and a block (or circuit) 192. The circuit 190 maybe implemented as a transcoding pipeline. The circuit 192 may beimplemented as an output buffer.

The apparatus 100 may provide live transcoding capable of frame accuratehot failover without observable discontinuities on the signal OUTPUT.The active input pipeline 160 is the input pipeline that is currentlyproviding multimedia data to the signal OUTPUT. The hot standby inputpipeline 162 is the backup pipeline. When a failover is detected, theinput pipeline that was previously the hot standby input pipeline 162becomes the new active input pipeline 160. Modifications to the delaybuffers 184 and/or 184′ will be described in connection with FIGS. 6 and7. After a failover, the previous active input pipeline 160 becomes thenew hot standby input pipeline 162.

The hot standby input pipeline 162 is normally configured to continuallyprovide demuxing, decoding, and/or buffering of data from the standbyinput SOURCE_2. Decoding the standby input SOURCE_2, even when thestandby input will not be passed to the signal OUTPUT, is beneficial forseveral reasons. For example, actively decoding the signal SOURCE_2allows the live transcode system 100 to actively monitor the health ofthe standby input signal SOURCE_2. Such monitoring may help inform theuser and/or the failover control logic 132 whether the standby inputsignal SOURCE_2 is ready to be made active if an interruption to thesignal SOURCE_1 occurs. In some embodiments, the health of the signalSOURCE_1 may also be actively monitored. The decoder 182 generates aseries of frames in response to the signal compressed bitstream SOURCE1. By decoding the standby input signal SOURCE_2, then buffering thoseframes, a hot failover (e.g., a fast failover) may be achieved.Pre-decoding the frames in the hot standby input pipeline 162 mayproduce a fast failover that may be covered with a smaller output buffer192.

The delay buffers 184 and/or 184′ are used to buffer the decoded framesreceived from the decoders 182 and/or 182′. The delay buffers 184 and184′ achieve two goals. The input failover is made frame accurate whenpossible. Alignment of the frames between the pipeline 160 and thepipeline 162 is determined by a plurality of input timecodes associatedwith the frames from each of the source signals SOURCE_1 and/orSOURCE_2. In general, one input timecode is associated with each framein each of the source signals SOURCE_1 and/or SOURCE_2. In one example,the timecodes may be implemented as SMPTE timecodes. However, othertimecodes may be implemented to meet the design criteria of a particularimplementation. Switching between the pipeline 160 and 162 should notcause buffer underfloor or overflow on the signal OUTPUT. Details of thebuffers 184 and/or 184′ will be described in more detail in connectionwith FIGS. 6 and 7.

Referring to FIG. 4, a block diagram of a method (or process) 200 isshown. The process 200 illustrates an overall operation of the device100. The process 200 generally comprises a step (or state) 202, a step(or state) 204, a step (or state) 206, a step (or state) 208, a step (orstate) 210, a step (or state) 212, a decision step (or state) 214, astep (or state) 216 and a step (or state) 218. An alternate branchgenerally comprises a step (or state) 206′, a step (or state) 208′, astep (or state) 210′, and a step (or state) 212′. The step 202 may be astart state. The step 204 may receive the redundant source signalsSOURCE_1 and SOURCE_2. The step 206 may demultiplex the signal SOURCE_1.The step 208 may decode the signal SOURCE_1. The state 210 may bufferthe frames decoded from the signal SOURCE_1. The step 212 may provideframe data to the output pipeline 170. The steps 206, 208, 210 and/or212 generally operate on the active pipeline 160 (e.g., the signalSOURCE_1). Similarly, the steps 206′, 208′, 210′ and 212′ generallyoperate on the standby pipeline 162 (e.g., the signal SOURCE_2). Thesteps 206′, 208′ and 210′ may be similar to the steps 206, 208 and 210.The step 212′ may actively monitor the health of the standby pipeline162. The decision step 214 may determine if a failure has occurred inthe active pipeline 160. if not, the method 200 moves to the step 218,which ends the method 200. If so, the method 200 moves to the state 216,which initiates a switch to the hot standby active source signalSOURCE_2. Next, the step 218 ends the method 200.

Referring to FIG. 5, a block diagram of a method (or process) 300 isshown. The method 300 generally describes the detection of the time codeand/or the delivery of the next frame during a transition after afailure. The method 300 generally comprises a step (or state) 302, astep (or state) 304, a decision step (or state) 306, a step (or state)308, a step (or state) 310, a decision step (or state) 312, a step (orstate) 314, a step (or state) 316, a step (or state) 318, and a step (orstate) 320. The step 302 may be a start state. The step 304 delivers anactive input. The decision state 306 determines if a failure hasoccurred. If not, the method 300 moves back to the state 304. If so, themethod 300 moves to the state 308. The state 308 inspects the inputtimecode of the last frame passed to the output pipeline 170. Next, thestate 310 checks the next hot standby frame in the hot standby delaybuffer 184′. Next, the decision state 312 determines if a hot standbyframe has a timecode less than or equal to the last active frametimecode. If so, the method 300 moves to the state 314, which drops aframe. If not, the method 300 moves to the state 316, which performs aninput switch. Next, the state 318 passes the next frame to the outputpipeline. The state 320 ends the method 300.

In some embodiments the timecode of frames may be used for selecting aproper frame to failover to. For example, the timecode of each frame maybe inspected. In another example, the timecode may be used to calculatea number of frames to drop at once. In some embodiments other methods oflining up frames from the active and/or standby input pipelines may beimplemented. In one example, an absolute sum of differences of thedecoded pictures on each pipeline may correlate pictures. The type ofmethod to determine which frame in the hot standby input pipeline shouldbe next (e.g., the frame to failover to) may be varied to meet thedesign criteria of a particular implementation.

Referring to FIG. 6, a block diagram of a method (or process) 400 isshown. The method 400 illustrates a process for resizing the delaybuffers 184 and/or 184′. The method 400 generally comprises a step (orstate) 402, a decision step (or state) 404, a step (or state) 406, astep (or state) 408, a step (or state) 410, a decision step (or state)412, a step (or state) 414, a step (or state) 416, a decision step (orstate) 418, a step (or state) 420 and a step (or state) 422.

The step 402 may be a start state. The decision step 404 may determineif a source failure has occurred. If so, the method 400 moves to thestate 406. If not, the method 400 moves to the step 408, which ends themethod 400. In the step 406, the method 400 retrieves a wall time of alast frame of a previous source and a first frame of a next source.Next, the state 410 determines a difference between the last frame ofthe previous source and the first frame of the next source. Next, thedecision state 412 determines if the active source led the hot standbysource. If so, the method 400 moves to the state 414, which makes arelative reduction to the delay buffer 184 or 184′ of the new activesource, then moves to the step 416. The step 416 completes the failoverprocess, then moves to the end step 408. If the decision step 412determines that the active source did not lead the standby source, themethod 400 moves to the decision state 418.

The decision state 418 determines if the active source lagged thestandby source. If so, the method 400 moves to the state 420, whichmakes a relative increase to the delay buffer (184 or 184′) of the newactive source. The method 400 then moves to the state 416, whichcompletes the source failover, then ends at the state 408. If thedecision state 418 determines that the active source did not lag thestandby source, the method 400 moves to the step 422, which keeps thesize of the delay buffer (184 or 184′) of the new active source thesame. The method 400 then moves to the step 416, which completes thefailover, then ends at the step 408.

Providing a frame accurate output without gaps is made possible by thedelay buffers 184 and/or 184′. The delay buffers 184 and 184′accommodate differences in network latency between the networkdelivering the active input bitstream SOURCE_1 and the networkdelivering the hot standby input bitstream SOURCE_2. When the activeinput SOURCE_1 is started, there is no guarantee that the hot standbyinput SOURCE_2 will be available. Therefore, the device 100 does notknow ahead of time whether the hot standby input signal SOURCE_1 willlead or lag the active input signal SOURCE_2 with respect to networklatency and/or arrival time.

Consider an example where the active input SOURCE_1 leads (e.g., arrivesearlier than) the hot standby input SOURCE_2 by 500 msec. Without thedelay buffer 184′, the desired frame in the pipeline 162 (e.g., theframe with a timecode one greater than previous frame passed to theoutput pipeline 170) would not be available on the hot standby input atthe time of an input switch. So, either the input switch would not beframe accurate or the encoder in the transcoding pipeline 170 would haveto wait 500 msec for the accurate frame to arrive, which could underflowthe signal OUTPUT. With a delay buffer 184 in the active input pipeline160 of >=500 msec, the frames from the decoder 182 are delayed so thatthe frame is immediately available on the hot standby input at the timeof the input switch.

Consider another example where the active input lags (arrives laterthan) the hot standby input by 500 msec. Without the delay buffer 184,the desired frame would have already been discarded before the inputswitch occurred. Therefore, the delay buffer 184′ is used on the hotstandby input pipeline 162. To achieve +/−500 msec of network tolerance,500 msec of delay on the active input and 2× that, or 1 second, of delayon the hot standby input is needed. The extra 500 msec compensates forthe 500 msec delay buffer 184 on the active input.

500 msec is used as an example. The particular amount of delayimplemented may be varied to meet the design criteria of a particularimplementation. In one example, a user configurable parameter may beused so that the operator may tune the maximum delay buffer occupancybased on the memory available in their system and the expected networklatency deviation between inputs.

The delay buffers 184 and/or 184′ help ensure the desired frame isavailable at the time of a failover. The system 100 still needs tochoose when to change inputs. When an input failover occurs, the systeminspects the input timecode of the last frame passed to the outputpipeline 170. Then frames in the hot standby delay buffer 184′ that havea timecode less than or equal to that timecode are dropped. The nextframe in that delay buffer 184′ will be the first one passed into theoutput pipeline 170 after the input switch.

Besides accommodating differences in network arrival time, the delaybuffers 184 and/or 184′ also give time for the failover control logic132 to monitor the inputs and determine that a failover is necessary.The delay buffers 184 and/or 184′ allow the live transcoder system 100to react and/or failover to the hot standby input SOURCE_2 before badand/or missing data detected by the input, demux, and/or decoder is everpassed into the output pipeline 170.

Input failover is not instantaneous since the device 100 has to teardown and re-establish part of the pipeline. In a standard livetranscoding system, there would be a danger of underflowing the outputbuffer during the failover. Avoiding underflowing during the failover isovercome via the output buffer 192 that has an occupancy initialized toa level that is greater than the amount that is expected to be drainedin the maximum time needed for an input switch to occur at the targetoutput bitrate. An independent high-priority thread responsible forpacing data out of the output buffer 192 to the output 144 ensures thatdata is paced from the output pipeline 170 during the input failover sothat there is not an observable discontinuity or timing anomaly on thesignal OUTPUT. Once the failover is complete, the occupancy of theoutput buffer 192 will be restored to the initial target occupancy bycarefully controlling the size of the delay buffers 184 and/or 184′ asdescribed below.

The device 100 generally ensures that the level of the output buffer 192is restored in a steady state. The occupancy of the output buffer 192 isultimately controlled by the input since (other than delays from thedelay buffers 184 and/or 184′) live inputs are processed as fast aspossible. The system 100 associates a system clock wall time with apresentation time stamp (PTS) of each input frame. In one example of aMPEG2 transport stream input over UDP, a system clock time may berecorded. Each program clock reference (PCR) is received. Each PTS isconverted into a timebase of the local system clock using an associationdescribed in EQ1:

WTsys=WTper+(PTS−PCR/300)  (EQ1)

The time WTsys may be a system clock wall time associated with the framethat has a presentation timestamp of PTS. The value PCR may be the lastprogram clock reference received. The time WTper may be the system clockwall time sampled when the system received the last program clockreference (PCR). In another example of a serial digital interface (SDI)input, the system clock may be sampled directly when the frame isreceived. The wall time associated with the PTS for other inputs may besimilarly derived

When an input switch (e.g., failover) occurs, the difference in walltime between the last frame of the previous input and the first frame ofthe next input may determine how much the output buffer 192 would changein size if no adjustment was made. Information from the difference inwall time between the last frame of the previous input and the firstframe of the next input may be used to resize the delay buffer 184and/or 184′ of the newly active input. For example, if the active inputleads the hot standby by 100 msec, then on an input switch the delaybuffer 184 and/or 184′ of the new input would be reduced by 100 msecrelative to the level of the previous active delay buffer to maintainthe previous output buffer occupancy.

An initial size of the delay buffer 184 (e.g., USER_MSEC) may be a userconfigurable parameter (e.g., a time specified in milliseconds). Theinitial size of the delay buffer 184′ may be a user configurableparameter set to two times the size of the delay buffer 184 (e.g.,2*USER_MSEC). In some embodiments, the initial size of the delay buffer184 and/or 184′ may be a default value. In some embodiments, the initialsize of the delay buffer 184 and/or 184′ may be a preset value. In otherembodiments, the initial size of the delay buffer 184 and/or 184′ may bedetermined based on a size of the memory 120 and/or the processingcapability of the processor 130. For example, the initial size of thedelay buffer 184 and/or 184′ may be determined independently of anyknowledge of the relative timing of the bitstreams (e.g., the hotstandby input SOURCE_2 may not be initially present). The total amountof buffering used (e.g., the total buffering of the delay buffer 184 andthe delay buffer 184′) may be 3*USER_MSEC.

During a failover transition both the delay buffer 184 and the delaybuffer 184′ may be re-sized. The size of the delay buffer 184 or 184′ inthe new active input pipeline may be determined (e.g., as described inconnection with FIGS. 6 and 7). The size of the delay buffer 184 or 184′in the new standby pipeline may be re-sized such that the total amountof buffering remains unchanged as shown in (EQ2):

NewDelayBufferStandby=(3*USER_MSEC)−NewDelayBufferActive  (EQ2)

Referring to FIGS. 7a-7c , operation of the device 100 with bufferoccupancy of the active input pipeline 160, the hot standby inputpipeline 162, and the output pipeline 170 is shown. Various frames areshown in the active input pipeline 160, the hot standby input pipeline162, and the output pipeline 170. The frames in the active inputpipeline 160 and the frames in the hot standby input pipeline 162 may bethe same frame (e.g., a frame having the same picture) but received froma different source. For example, the frames in the active input pipeline160 may be frames received from SOURCE_1 and the frames in the hotstandby input pipeline 162 may be frames received from SOURCE_2.

The frames in the output pipeline 170 may be received from the activeinput pipeline 160 or the hot standby pipeline 162. For example, eithera frame from the active input pipeline 160 or a frame from the hotstandby output pipeline 162 may be sent to the output pipeline 170. Eachframe may have an associated value WTsys. The value WTsys may be asystem clock wall time associated with the presentation timestamp ofeach frame (e.g., shown as t+4 through t−3).

The hot standby pipeline 162 is shown having a buffer depth of 4 greaterthan the active input pipeline 160. For example, the hot standbypipeline 162 may have a buffer depth of 8 (e.g., in the delay buffer184′) and the active input pipeline may have a buffer depth of 4 (e.g.,in the delay buffer 184). The depth of the delay buffer 184 and/or thedelay buffer 184′ may be varied according to the design criteria of aparticular implementation.

Each frame is shown having a subscript label. The subscript label of theframe may indicate whether a frame has the same picture as another frame(e.g., identify the frame). In one example, the subscript label of theframe (e.g., n−3, n+1, etc.) may represent a timecode of the frame.Frames having the same subscript label may represent the same picture.For example, the frame An+4 may represent the same picture as the frameSn+4 but may be provided through a different source (e.g., the frameAn+4 may be received by SOURCE_1 and the frame Sn+4 may be received bySOURCE_2).

Referring to FIG. 7a , buffer occupancy when the arrival of frames ismatched is shown (e.g., the arrival of SOURCE_1 at the input 140 and thearrival of SOURCE_2 at the input 142 are matched). The active inputpipeline 160 is shown having a number of active frames An+4 throughAn+1. The hot standby pipeline 162 is shown having a number of framesSn+4 through Sn−3. The output pipeline 170 is shown having frames On−1through On−4. The frame An+0 is shown being presented by the activeinput pipeline 160 to the output pipeline 170. If no failure occurs, thenext frame queued up to be passed to the output pipeline 170 would bethe frame An+1. The WTsys of the frame An+1 is shown as t+1.

If a failover happened at this moment (e.g., the active input pipeline160 was unable to send the frame An+1 to the output pipeline 170), theframe Sn+1 would need to be sent to the output pipeline 170 to ensureframe accuracy. The frames Sn+0 through Sn−3 in the hot standby inputpipeline 162 would be dropped (e.g., the corresponding frames On−1through On−3 and An+0 have already been sent to the output pipeline170).

The next frame, Sn+1, is shown having a WTsys of t+1. The WTsys of theframe Sn+1 and the frame An+1 are equal. The pipeline 162 will becomethe new active input pipeline, and the depth of the delay buffers 184and/or 184′ may be re-sized. Since the WTsys of the frames Sn+1 and theframe An+1 are equal, the buffer depth of the pipeline 162 may bere-sized to be the same size as the delay buffer of the previous activepipeline 160 (e.g., a buffer depth of 4).

Referring to FIG. 7b , buffer occupancy when the arrival of frames inthe active input pipeline 160 leads the arrival of frames in the hotstandby pipeline 162 (e.g., the arrival of SOURCE_i at the input 140leads the arrival of SOURCE_2 at the input 142 by two frame times) isshown. The active input pipeline 160 is shown having a number of activeframes An+4 through An+1. The hot standby pipeline 162 is shown having anumber of frames Sn+2 through Sn−5. The output pipeline 170 is shownhaving frames On−1 through On−4. The frame An+0 is shown being sent fromthe active input pipeline 160 to the output pipeline 170. If no failureoccurs, the next frame queued up to be passed to the output pipeline 170would be the frame An+1. The WTsys of the frame An+1 is shown as t+1.

If a failover happened at this moment (e.g., the active input pipeline160 was unable to send the frame An+1 to the output pipeline 170), theframe Sn+1 would need to be sent to the output pipeline 170 to ensureframe accuracy. The frames Sn+0 through Sn−5 in the hot standby inputpipeline 162 would be dropped (e.g., the corresponding frames On−1through On−5 and An+0 have already been sent to the output pipeline170).

The next frame, Sn+1, is shown having a WTsys of t+3. The WTsys of theframe Sn+1 and the frame An+1 are not equal and An+1 leads Sn+1. Thepipeline 162 will become the new active input pipeline, and the depth ofthe delay buffers 184 and/or 184′ may be re-sized. Since the WTsys ofthe frame Sn+1 and the frame An+1 have a difference of 2 with the frameAn+1 leading the arrival of the frame Sn+1, the buffer depth of thepipeline 162 may be re-sized to be the size of the delay buffer 184 inthe previous active input pipeline 160 minus two frames (e.g., a bufferdepth of 2). Decreasing the size of the delay buffer 184′ after thefailover occurs relative to the occupancy of the delay buffer 184 priorto the failover may compensate for the later arrival of frames fromSOURCE_2 relative to SOURCE_1 (e.g., to ensure the output buffer 192does not underfloor).

Referring to FIG. 7c , buffer occupancy when the arrival of frames inthe active input pipeline 160 lags the arrival of frames in the hotstandby pipeline 162 (e.g., the arrival of SOURCE_i at the input 140lags the arrival of SOURCE_2 at the input 142 by two frame times) isshown. The active input pipeline 160 is shown having a number of activeframes An+4 through An+1. The hot standby pipeline 162 is shown having anumber of frames Sn+6 through Sn−i. The output pipeline 170 is shownhaving frames On−1 through On−4. The frame An+0 is shown being sent fromthe active input pipeline 160 to the output pipeline 170. If no failureoccurs, the next frame queued up to be passed to the output pipeline 170would be the frame An+1. The WTsys of the frame An+1 is shown as t+1.

If a failover happened at this moment (e.g., the active input pipeline160 was unable to send the frame An+1 to the output pipeline 170), theframe Sn+1 would need to be sent to the output pipeline 170 to ensureframe accuracy. The frames Sn−1 and Sn+0 in the hot standby inputpipeline 162 would be dropped (e.g., the corresponding frames On−1 andAn+0 have already been sent to the output pipeline 170).

The next frame, Sn+1, is shown having a WTsys of t−1. The WTsys of theframe Sn+1 and the frame An+1 are not equal and An+1 lags Sn+1. Thepipeline 162 will become the new active input pipeline, and the depth ofthe delay buffers 184 and/or 184′ may be re-sized. Since the WTsys ofthe frame Sn+1 and the frame An+1 have a difference of 2 with the frameAn+1 lagging the arrival of the frame Sn+1, the buffer depth of thepipeline 162 may be re-sized to be the size of the delay buffer 184 inthe previous active input pipeline 160 plus two frames (e.g., a bufferdepth of 6). Increasing the size of the delay buffer 184′ after thefailover occurs relative to the occupancy of the delay buffer 184 priorto the failover may compensate for the earlier arrival of frames fromSOURCE_2 relative to SOURCE_1 (e.g., to ensure the output buffer 192does not overflow).

Referring to FIG. 8 an alternate detailed diagram of the apparatus 100′is shown. The apparatus 100′ is shown comprising the memory 120′. Thememory 120′ is shown comprising an active input pipeline 160′, a hotstandby pipeline 162′ and an output pipeline 170′. The active inputpipeline 160′ generally comprises a block (or circuit) 480, a block (orcircuit) 482 and a block (or circuit) 484. The circuit 480 may beimplemented as the demultiplexing circuit. The circuit 482 may beimplemented as the delay buffer. The circuit 484 may be implemented asthe decoder circuit. Similarly, the pipeline 162′ may have a block (orcircuit) 480′, a block (or circuit) 482′ and a block (or circuit) 484′.The circuits 480′, 482′ and 484′ may have similar implementations as thecircuits 480, 482 and 484. The output pipeline 170′ generally comprisesa block (or circuit) 490 and a block (or circuit) 492. The circuit 490may be implemented as the transcoding pipeline. The circuit 492 may beimplemented as the output buffer.

In the embodiment shown in FIG. 3, the delay buffers 184 and 184′ areshown after the decoders 182 and 182′. In the embodiment shown in FIG.8, the device 100′ has the delay buffers 482 and 482′ between the demux480 (or 480′) and the decoder 484 (or 484′). In such an implementation,the inputs are often compressed and thus take less memory to bufferprior to the decoders 484 and 484′. The embodiment shown in the device100 and the embodiment shown in the device 100′ may have similar latencyand/or fast failover characteristics.

The output of the delay buffer 482′ in the hot standby input pipeline162′ needs to monitor pictures being passed into the output pipeline170′ by the active input pipeline 160′ to ensure that the hot standbyinput keeps up with the active input. That is, if the active inputdecodes and passes to the output pipeline 170′ a picture with timecodeT, then the hot standby input pipeline 162′ should decode and discardall pictures with timecode less than or equal to T. Failing to do sowould mean that at the time of the failover, even though the targetframe should be in the hot standby input delay buffer 482′ it could beproceeded by many other frames that (due to the nature of long-GOPreference pictures) would need to be decoded in order to successfullydecode the target frame. The processing time required to decode thoseextra pictures could result in an underflow of the output buffer 492.

The device 100 and/or 100′ provides a frame accurate hot failover of alive transcoding system from one multimedia input to an alternatemultimedia input without any observable audio-visual discontinuities ortiming anomalies on the output.

The functions performed by the diagrams of FIGS. 4-6 may be implementedusing one or more of a conventional general purpose processor, digitalcomputer, microprocessor, microcontroller, RISC (reduced instruction setcomputer) processor, CISC (complex instruction set computer) processor,SIMD (single instruction multiple data) processor, signal processor,central processing unit (CPU), arithmetic logic unit (ALU), videodigital signal processor (VDSP) and/or similar computational machines,programmed according to the teachings of the specification, as will beapparent to those skilled in the relevant art(s). Appropriate software,firmware, coding, routines, instructions, opcodes, microcode, and/orprogram modules may readily be prepared by skilled programmers based onthe teachings of the disclosure, as will also be apparent to thoseskilled in the relevant art(s). The software is generally executed froma medium or several media by one or more of the processors of themachine implementation.

The invention may also be implemented by the preparation of ASICs(application specific integrated circuits), Platform ASICs, FPGAs (fieldprogrammable gate arrays), PLDs (programmable logic devices), CPLDs(complex programmable logic devices), sea-of-gates, RFICs (radiofrequency integrated circuits), ASSPs (application specific standardproducts), one or more monolithic integrated circuits, one or more chipsor die arranged as flip-chip modules and/or multi-chip modules or byinterconnecting an appropriate network of conventional componentcircuits, as is described herein, modifications of which will be readilyapparent to those skilled in the art(s).

The invention thus may also include a computer product which may be astorage medium or media and/or a transmission medium or media includinginstructions which may be used to program a machine to perform one ormore processes or methods in accordance with the invention. Execution ofinstructions contained in the computer product by the machine, alongwith operations of surrounding circuitry, may transform input data intoone or more files on the storage medium and/or one or more outputsignals representative of a physical object or substance, such as anaudio and/or visual depiction. The storage medium may include, but isnot limited to, any type of disk including floppy disk, hard drive,magnetic disk, optical disk, CD-ROM, DVD and magneto-optical disks andcircuits such as ROMs (read-only memories), RAMS (random accessmemories), EPROMs (erasable programmable ROMs), EEPROMs (electricallyerasable programmable ROMs), UVPROM (ultra-violet erasable programmableROMs), Flash memory, magnetic cards, optical cards, and/or any type ofmedia suitable for storing electronic instructions.

The elements of the invention may form part or all of one or moredevices, units, components, systems, machines and/or apparatuses. Thedevices may include, but are not limited to, servers, workstations,storage array controllers, storage systems, personal computers, laptopcomputers, notebook computers, palm computers, personal digitalassistants, portable electronic devices, battery powered devices,set-top boxes, encoders, decoders, transcoders, compressors,decompressors, pre-processors, post-processors, transmitters, receivers,transceivers, cipher circuits, cellular telephones, digital cameras,positioning and/or navigation systems, medical equipment, heads-updisplays, wireless devices, audio recording, audio storage and/or audioplayback devices, video recording, video storage and/or video playbackdevices, game platforms, peripherals and/or multi-chip modules. Thoseskilled in the relevant art(s) would understand that the elements of theinvention may be implemented in other types of devices to meet thecriteria of a particular application.

While the invention has been particularly shown and described withreference to the preferred embodiments thereof, it will be understood bythose skilled in the art that various changes in form and details may bemade without departing from the scope of the invention.

1. A computer-implemented method comprising: receiving a first bitstreamfor a first pipeline and a second bitstream for a second pipeline;sending an output of the first pipeline to an output pipeline;determining an error has occurred in the first pipeline; and sending anoutput that is associated with an adjustment frame of the secondpipeline to the output pipeline.
 2. The computer-implemented method ofclaim 1, wherein the first bitstream and the second bitstream compriseredundant information received from independent sources.
 3. Thecomputer-implemented method of claim 1, further comprising: adjusting afirst size of a first buffer of the first pipeline and a second size ofa second buffer of the second pipeline based at least on timinginformation associated with the first bitstream and the secondbitstream.
 4. The computer-implemented method of claim 1, furthercomprising: identifying a first timecode associated with a last framesent to the output pipeline from the first pipeline; identifying asecond timecode associated with a next frame in a buffer associated withthe second pipeline; determining the second timecode is less than orequal to the first timecode; and dropping the next frame.
 5. Thecomputer-implemented method of claim 1, further comprising: identifyinga first timecode associated with a last frame sent to the outputpipeline from the first pipeline; identifying a second timecodeassociated with a next frame in a buffer associated with the secondpipeline; determining the second timecode is greater than the firsttimecode; and sending the next frame to the output pipeline.
 6. Thecomputer-implemented method of claim 1, further comprising: identifyinga first timecode associated with a last frame sent to the outputpipeline from the first pipeline; identifying a second timecodeassociated with a next frame in a buffer associated with the secondpipeline; determining the second timecode is greater than the firsttimecode; and increasing the size of the buffer based at least on adifference between the second timecode and the first timecode.
 7. Thecomputer-implemented method of claim 1, further comprising: identifyinga first timecode associated with a last frame sent to the outputpipeline from the first pipeline; identifying a second timecodeassociated with a next frame in a buffer associated with the secondpipeline; determining the second timecode is less than the firsttimecode; and decreasing the size of the buffer based at least on adifference between the second timecode and the first timecode.
 8. Thecomputer-implemented method of claim 1, further comprising: identifyinga first timecode associated with a last frame sent to the outputpipeline from the first pipeline; identifying a second timecodeassociated with a next frame in a buffer associated with the secondpipeline; determining the second timecode is equal to the firsttimecode; and maintaining the size of the buffer.
 9. Thecomputer-implemented method of claim 1, wherein the first pipeline andthe second pipeline are each configured to perform one or more ofdemultiplexing, decoding, or buffering.
 10. The computer-implementedmethod of claim 1, wherein the output pipeline comprises an outputbuffer having a size based at least on an output bitrate and a time toswitch from the output of the first pipeline and the output of thesecond pipeline.
 11. A system comprising: at least one processor; and atleast one computer readable storage medium including instructions storedthereon which, when executed, cause the system to: receive a firstbitstream for a first pipeline and a second bitstream for a secondpipeline; send an output of the first pipeline to an output pipeline;determine an error has occurred in the first pipeline; and send anoutput that is associated with an adjustment frame of the secondpipeline to the output pipeline.
 12. The system of claim 11, wherein theinstructions, when executed, further cause the system to: adjust a firstsize of a first buffer of the first pipeline and a second size of asecond buffer of the second pipeline based at least on timinginformation associated with the first bitstream and the secondbitstream.
 13. The system of claim 11, wherein the instructions, whenexecuted, further cause the system to: identify a first timecodeassociated with a last frame sent to the output pipeline from the firstpipeline; identify a second timecode associated with a next frame in abuffer associated with the second pipeline; determine the secondtimecode is less than or equal to the first timecode; and drop the nextframe.
 14. The system of claim 11, wherein the instructions, whenexecuted, further cause the system to: identify a first timecodeassociated with a last frame sent to the output pipeline from the firstpipeline; identify a second timecode associated with a next frame in abuffer associated with the second pipeline; determine the secondtimecode is greater than the first timecode; and send the next frame tothe output pipeline.
 15. The system of claim 11, wherein theinstructions, when executed, further cause the system to: identify afirst timecode associated with a last frame sent to the output pipelinefrom the first pipeline; identify a second timecode associated with anext frame in a buffer associated with the second pipeline; determinethe second timecode is greater than the first timecode; and increase thesize of the buffer based at least on a difference between the secondtimecode and the first timecode.
 16. The system of claim 11, wherein theinstructions, when executed, further cause the system to: identify afirst timecode associated with a last frame sent to the output pipelinefrom the first pipeline; identify a second timecode associated with anext frame in a buffer associated with the second pipeline; determinethe second timecode is less than the first timecode; and decrease thesize of the buffer based at least on a difference between the secondtimecode and the first timecode.
 17. The system of claim 11, wherein theinstructions, when executed, further cause the system to: identify afirst timecode associated with a last frame sent to the output pipelinefrom the first pipeline; identify a second timecode associated with anext frame in a buffer associated with the second pipeline; determinethe second timecode is equal to the first timecode; and maintain thesize of the buffer.
 18. The system of claim 11, wherein the outputpipeline comprises an output buffer having a size based at least on anoutput bitrate and a time to switch from the output of the firstpipeline and the output of the second pipeline.
 19. An apparatuscomprising: a first pipeline configured to receive a first bitstream; asecond pipeline configured to receive a second bitstream; an outputpipeline configured to receive an output from either of the firstpipeline or the second pipeline; and at least one processor configuredto: determine an error has occurred in the first pipeline; and send anoutput of the second pipeline to the output pipeline.
 20. The apparatusof claim 19, wherein the processor is further configured to: adjust afirst size of a first buffer of the first pipeline and a second size ofa second buffer of the second pipeline based at least on timinginformation associated with the first bitstream and the secondbitstream.